New Project > Select Repository #4443

AlexTugarev · 2021-06-09T03:30:40Z

This PR introduces the project creation flow (gitpod.io/new)

GH App Change

⚠️ Please note, this change requires a significant GH App config change to be aligned with any deployment: instead of redirecting to an URL of the dashboard, it should redirect to the /api/apps/github/setup route introduced here.

Preview environment

The URL is https://at-select-repo.staging.gitpod-dev.com/new

⚠️ Please note, this preview environment requires an additional re-deployment with werft from within a Gitpod workspace using werft run github -s chart/values.yaml -f, where the provided values should contain the GH App config. In addition, a secret with the private key of the GH App should be created like this kubectl create secret generic server-github-app-cert ...

New Project

The drop-down list contains the user's account by default. If the GH App is installed on Orgs of the users and the user granted read:org permissions, these accounts should be visible as well.

This list should be filtered to exclude repositories already in used in any project.

Select Git Provider

Reconfigure GH App installations

You will see the GH App configuration in a pop-up window. For the preview environment this is a test app of mine.

Select a team

Listing of all teams of a user.

Incorporate "New Team" inline

List projects of a team

This is a starting point for the Projects overview.

Closes #4352

components/server/ee/src/prebuilds/github-app.ts

AlexTugarev · 2021-06-14T04:04:54Z

components/server/src/workspace/gitpod-server-impl.ts

+
+    //#region Projects concerns
+    //
+    async getProviderRepositoriesForUser(params: { provider: string }): Promise<ProviderRepository[]> {


this is implemented in EE space because the github-app.ts is also there.

@svenefftinge, it's ok to move the github-app.ts to the non-EE sources?

components/server/ee/src/prebuilds/github-app.ts

gtsiolis

Great to see this coming to life @AlexTugarev! Left some minor UX comments below. 🏀

components/dashboard/src/projects/NewProject.tsx

gtsiolis · 2021-06-14T07:57:01Z

components/dashboard/src/projects/NewProject.tsx

+                </div>
+                <div className="p-3 bg-gray-100">
+                    <div className="text-gray-500 text-center">
+                        Repository not found? <a href="javascript:void(0)" onClick={e => reconfigure()} className="text-gray-400 underline underline-thickness-thin underline-offset-small hover:text-gray-600">Reconfigure</a>


question: Is it possible to switch to the org where the app has been installed after installing?

Yes.

When selecting Reconfigure and editing the installations of the App on GH, you may have either:

Install on new Account/Org

Selected additional Repos (or selected all)

Deleted the installation from an Account/Org

All these kinds of changes needs to be reflected in the action (Reconfigure) and also updated after the action is finished, which I doubt happens in all cases. At least after deletion, we're not getting redirected :-( So in that case, where the user does (need to) close the GH pop-up window, we need to update!

gtsiolis · 2021-06-14T07:58:16Z

components/dashboard/src/projects/NewProject.tsx

+                                        </div>
+                                        <div className="w-4/12 flex justify-end">
+                                            <div className="flex self-center hover:bg-gray-200 dark:hover:bg-gray-700 rounded-md cursor-pointer opacity-0 group-hover:opacity-100">
+                                                <button className="primary py-1" onClick={() => selectRepo(r)}>Select</button>


thought: If we don't use any other label for this button we could remove it and use the whole div as a hyperlink as done with similar modals (see new workspace, parallel workspaces, etc).

would that apply to the Select Team page as well?

In case the project is already part of a team we could trigger a modal with a CTA button to request access to the team, instead of having conditional buttons here Add Project and Select Team. WDYT?

components/dashboard/src/projects/NewProject.tsx

AlexTugarev · 2021-06-15T06:27:11Z

#4443 (comment)

In case the project is already part of a team we could trigger a modal with a CTA button to request access to the team, instead of having conditional buttons here Add Project and Select Team. WDYT?

@gtsiolis, thanks for bringing this up! It's a rather conceptual question, so let's discuss.

If a repository (aka project) is already linked to a team, what is the implication to the overall concept and the creation flow?Potentially incomplete list of options:

Exclude taken repositories on Create Project page.
Disable taken repositories on Create Project page.
Allow every accessible repo on Create Project page, then add an invitation method.

svenefftinge · 2021-06-15T06:50:03Z

/werft run

👍 started the job as gitpod-build-at-select-repo.40

AlexTugarev · 2021-06-15T12:19:46Z

The check for the state parameter on setup callbacks provides a nice migration path. When installing the app from GH, we get redirected to /install-github-app like before

svenefftinge · 2021-06-16T08:12:15Z

When I add an org, it replaces the org I previously added.
Even when I already have a project for an org, I don't see that org anymore in the list.
When I land on new, nothing in the combo box is selected.
When I added an app installation, the newly added installation is not selected
When I have added the 'orgs' scope, I still don't see the orgs for which installations exist

svenefftinge · 2021-06-16T08:53:40Z

The new project flow should not have any menus or so. See design docs. But we'll need to allow users to abort the flow (cc @gtsiolis )

AlexTugarev · 2021-06-16T09:37:17Z

When I have added the 'orgs' scope, I still don't see the orgs for which installations exist

@svenefftinge, that is true as long as the gitpod-dev OAuth app isn't approved for the org. On https://at-select-repo.staging.gitpod-dev.com/integrations you can select Manage github.com or directly navigate to
https://github.com/settings/connections/applications/51329bab66290acdf39f and make sure you can approve an organization to test with.

AlexTugarev · 2021-06-18T09:03:21Z

@svenefftinge, I updated the PR with following changes:

When I add an org, it replaces the org I previously added.

I'm not sure, what that means. The Reconfigure action (showing GitHub's installation page) combines several options: installing, updating, or deleting.

Even when I already have a project for an org, I don't see that org anymore in the list.

That should be solved now. knownOrgs are preserved in User.additionalData, so that when you come back later to /new, you should be able to browse the repos of that org.

When I land on new, nothing in the combo box is selected.

That might be just related to the missing org, I suppose. In general it takes up to seconds if you have many orgs accessible to the OAuth App of the Gitpod installation.

When I added an app installation, the newly added installation is not selected

Done. They should be selected now properly. Also adding updateAt for installations and repositories. Repositories are sorted now.

When I have added the 'orgs' scope, I still don't see the orgs for which installations exist

This is fixed in the sense that known orgs are preserved on backend now.

AlexTugarev · 2021-06-18T09:07:27Z

The new project flow should not have any menus or so. See design docs.

@svenefftinge and @gtsiolis, I aligned with /new-teams on that.

I'm happy to reduce the menus. For both?

svenefftinge

Works well. It needs additional passes, of course, but since it is not activated / visible by default it would be good to merge ASAP. So we can iterate in smaller batches.

svenefftinge · 2021-06-22T11:29:22Z

components/gitpod-db/src/typeorm/migration/1622468446118-TeamsAndProjects.ts

@@ -11,7 +11,7 @@ export class TeamsAndProjects1622468446118 implements MigrationInterface {
    public async up(queryRunner: QueryRunner): Promise<any> {
        await queryRunner.query("CREATE TABLE IF NOT EXISTS `d_b_team` (`id` char(36) NOT NULL, `name` varchar(255) NOT NULL, `slug` varchar(255) NOT NULL, `creationTime` varchar(255) NOT NULL, `deleted` tinyint(4) NOT NULL DEFAULT '0', `_lastModified` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), PRIMARY KEY (`id`), KEY `ind_dbsync` (`_lastModified`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;");
        await queryRunner.query("CREATE TABLE IF NOT EXISTS `d_b_team_membership` (`id` char(36) NOT NULL, `teamId` char(36) NOT NULL, `userId` char(36) NOT NULL, `role` varchar(255) NOT NULL, `creationTime` varchar(255) NOT NULL, `deleted` tinyint(4) NOT NULL DEFAULT '0', `_lastModified` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), PRIMARY KEY (`id`), KEY `ind_teamId` (`teamId`), KEY `ind_userId` (`userId`), KEY `ind_dbsync` (`_lastModified`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;");
-        await queryRunner.query("CREATE TABLE IF NOT EXISTS `d_b_project` (`id` char(36) NOT NULL, `cloneUrl` varchar(255) NOT NULL, `teamId` char(36) NOT NULL, `appInstallationId` varchar(255) NOT NULL, `creationTime` varchar(255) NOT NULL, `deleted` tinyint(4) NOT NULL DEFAULT '0', `_lastModified` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), PRIMARY KEY (`id`), KEY `ind_teamId` (`teamId`), KEY `ind_dbsync` (`_lastModified`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;");
+        await queryRunner.query("CREATE TABLE IF NOT EXISTS `d_b_project` (`id` char(36) NOT NULL, `name` varchar(255) NOT NULL, `cloneUrl` varchar(255) NOT NULL, `teamId` char(36) NOT NULL, `appInstallationId` varchar(255) NOT NULL, `creationTime` varchar(255) NOT NULL, `deleted` tinyint(4) NOT NULL DEFAULT '0', `_lastModified` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), PRIMARY KEY (`id`), KEY `ind_teamId` (`teamId`), KEY `ind_dbsync` (`_lastModified`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;");


do you plan to manually update the staging DB?

good point! this is already on staging.
yes, it would be best to do a manual update.

svenefftinge · 2021-06-22T11:30:58Z

components/gitpod-protocol/src/protocol.ts

@@ -1205,6 +1208,27 @@ export interface Project {
    deleted?: boolean;
 }

+export interface ProjectInfo {


Why copying the whole Project shape instead of referencing it?

Just learned that DBProject isn't implementing Project at all.
So, yeah it doesn't make sense to duplicate.

Besides that, we introduced xInfo shapes to augment entities, to make it works nicer with the frontend.

svenefftinge · 2021-06-22T11:34:54Z

components/server/ee/src/workspace/gitpod-server-impl.ts

+    //#region Projects concerns
+    //
+    async getProviderRepositoriesForUser(params: { provider: string, hints?: object }): Promise<ProviderRepository[]> {
+        const user = this.checkAndBlockUser("getProviderRepositoriesForUser");


this method should go the respective GitProvider impl. We should not add dependencies to GitHub (octokit) across our code.

+1 for moving out. I'm thinking of a GitHubAppSupport injectable.

FYI we have two octokit dependencies

first in our GitHub support (auth provider, context parsers, etc.)

second comes in via probot
Both parts live in different contexts. More specifically: the GitHub support is instantiated per Git Provider host.

BTW it's ~~impossible~~ impractical to sync both octokit versions both. Also, obtaining GH App's authenticated API (octokit) is done via GitHubApp.server.probotApp.auth().

jankeromnes · 2021-06-23T08:22:17Z

components/gitpod-db/src/typeorm/project-db-impl.ts

+
+        const project: Project = {
+            id: uuidv4(),
+            name,


Nit: Since name is also used as a slug, maybe validate it here against a regex? For example like so (maybe without spaces):

gitpod/components/gitpod-db/src/typeorm/team-db-impl.ts

Lines 74 to 76 in 903c854

if (!/^[A-Za-z0-9 _-]+$/.test(name)) {

throw new Error('Please choose a team name containing only letters, numbers, -, _, or spaces.');

}

AlexTugarev force-pushed the at/select-repo branch 10 times, most recently from 4f6d80b to 0d15bc6 Compare June 13, 2021 11:20

AlexTugarev commented Jun 14, 2021

View reviewed changes

components/server/ee/src/prebuilds/github-app.ts Outdated Show resolved Hide resolved

AlexTugarev commented Jun 14, 2021

View reviewed changes

AlexTugarev changed the title ~~WIP New Project > Select Repository~~ New Project > Select Repository Jun 14, 2021

AlexTugarev commented Jun 14, 2021

View reviewed changes

components/server/ee/src/prebuilds/github-app.ts Show resolved Hide resolved

gtsiolis reviewed Jun 14, 2021

View reviewed changes

AlexTugarev force-pushed the at/select-repo branch 2 times, most recently from 45d6156 to 9e6aaab Compare June 14, 2021 14:09

AlexTugarev force-pushed the at/select-repo branch 3 times, most recently from 9366cd1 to 1bb1af2 Compare June 15, 2021 09:53

svenefftinge added the roadmap item: teams & projects label Jun 16, 2021

AlexTugarev force-pushed the at/select-repo branch 2 times, most recently from 4c50602 to a61e38a Compare June 18, 2021 08:32

AlexTugarev marked this pull request as ready for review June 18, 2021 14:04

AlexTugarev requested a review from jankeromnes June 21, 2021 13:16

AlexTugarev force-pushed the at/select-repo branch from 195d90a to 255d021 Compare June 22, 2021 11:28

svenefftinge approved these changes Jun 22, 2021

View reviewed changes

jankeromnes reviewed Jun 23, 2021

View reviewed changes

gtsiolis added the component: dashboard label Jun 23, 2021

AlexTugarev force-pushed the at/select-repo branch 2 times, most recently from e961ad6 to 0fa4916 Compare June 23, 2021 12:55

Add New Project page and GH App installation

5cab5b8

AlexTugarev force-pushed the at/select-repo branch from 0fa4916 to 5cab5b8 Compare June 23, 2021 13:25

AlexTugarev merged commit 8552d9e into main Jun 23, 2021

AlexTugarev deleted the at/select-repo branch June 23, 2021 13:36

gtsiolis mentioned this pull request Jul 6, 2021

Fix dark theme colors and overflowing rounded corners for the project import pages #4717

Closed

This was referenced Aug 5, 2021

Redirect new users to /new #5071

Closed

Add support for importing GitLab projects #5078

Closed

gtsiolis mentioned this pull request Sep 6, 2021

Change provider dropdown icon size #5545

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Project > Select Repository #4443

New Project > Select Repository #4443

AlexTugarev commented Jun 9, 2021 •

edited

Loading

AlexTugarev Jun 14, 2021

gtsiolis left a comment

gtsiolis Jun 14, 2021

AlexTugarev Jun 14, 2021

gtsiolis Jun 14, 2021

AlexTugarev Jun 14, 2021

gtsiolis Jun 14, 2021

AlexTugarev commented Jun 15, 2021

svenefftinge commented Jun 15, 2021 •

edited by werft-gitpod-dev-com bot

Loading

AlexTugarev commented Jun 15, 2021

svenefftinge commented Jun 16, 2021 •

edited

Loading

svenefftinge commented Jun 16, 2021 •

edited

Loading

AlexTugarev commented Jun 16, 2021

AlexTugarev commented Jun 18, 2021

AlexTugarev commented Jun 18, 2021

svenefftinge left a comment

svenefftinge Jun 22, 2021

AlexTugarev Jun 22, 2021

svenefftinge Jun 22, 2021

AlexTugarev Jun 22, 2021

svenefftinge Jun 22, 2021

AlexTugarev Jun 22, 2021

jankeromnes Jun 23, 2021 •

edited

Loading

	if (!/^[A-Za-z0-9 _-]+$/.test(name)) {
	throw new Error('Please choose a team name containing only letters, numbers, -, _, or spaces.');
	}

New Project > Select Repository #4443

New Project > Select Repository #4443

Conversation

AlexTugarev commented Jun 9, 2021 • edited Loading

This PR introduces the project creation flow (gitpod.io/new)

GH App Change

Preview environment

New Project

Select Git Provider

Reconfigure GH App installations

Select a team

List projects of a team

Choose a reason for hiding this comment

gtsiolis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexTugarev commented Jun 15, 2021

svenefftinge commented Jun 15, 2021 • edited by werft-gitpod-dev-com bot Loading

AlexTugarev commented Jun 15, 2021

svenefftinge commented Jun 16, 2021 • edited Loading

svenefftinge commented Jun 16, 2021 • edited Loading

AlexTugarev commented Jun 16, 2021

AlexTugarev commented Jun 18, 2021

AlexTugarev commented Jun 18, 2021

svenefftinge left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jankeromnes Jun 23, 2021 • edited Loading

Choose a reason for hiding this comment

AlexTugarev commented Jun 9, 2021 •

edited

Loading

svenefftinge commented Jun 15, 2021 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Jun 16, 2021 •

edited

Loading

svenefftinge commented Jun 16, 2021 •

edited

Loading

jankeromnes Jun 23, 2021 •

edited

Loading